Evolving machine learning and deep learning models using evolutionary algorithms

Despite the great success in data mining, machine learning and deep learning models are yet subject to material obstacles when tackling real-life challenges, such as feature selection, initialization sensitivity, as well as hyperparameter optimization. The prevalence of these obstacles has severely constrained conventional machine learning and deep learning methods from fulfilling their potentials. In this research, three evolving machine learning and one evolving deep learning models are proposed to eliminate above bottlenecks, i.e. improving model initialization, enhancing feature representation, as well as optimizing model configuration, respectively, through hybridization between the advanced evolutionary algorithms and the conventional ML and DL methods. Specifically, two Firefly Algorithm based evolutionary clustering models are proposed to optimize cluster centroids in K-means and overcome initialization sensitivity as well as local stagnation. Secondly, a Particle Swarm Optimization based evolving feature selection model is developed for automatic identification of the most effective feature subset and reduction of feature dimensionality for tackling classification problems. Lastly, a Grey Wolf Optimizer based evolving Convolutional Neural Network-Long Short-Term Memory method is devised for automatic generation of the optimal topological and learning configurations for Convolutional Neural Network-Long Short-Term Memory networks to undertake multivariate time series prediction problems. Moreover, a variety of tailored search strategies are proposed to eliminate the intrinsic limitations embedded in the search mechanisms of the three employed evolutionary algorithms, i.e. the dictation of the global best signal in Particle Swarm Optimization, the constraint of the diagonal movement in Firefly Algorithm, as well as the acute contraction of search territory in Grey Wolf Optimizer, respectively. The remedy strategies include the diversification of guiding signals, the adaptive nonlinear search parameters, the hybrid position updating mechanisms, as well as the enhancement of population leaders. As such, the enhanced Particle Swarm Optimization, Firefly Algorithm, and Grey Wolf Optimizer variants are more likely to attain global optimality on complex search landscapes embedded in data mining problems, owing to the elevated search diversity as well as the achievement of advanced trade-offs between exploration and exploitation

Evolving CNN-LSTM Models for Time Series Prediction Using Enhanced Grey Wolf Optimizer

In this research, we propose an enhanced Grey Wolf Optimizer (GWO) for designing the evolving Convolutional Neural Network-Long Short-Term Memory (CNN-LSTM) networks for time series analysis. To overcome the probability of stagnation at local optima and a slow convergence rate of the classical GWO algorithm, the newly proposed variant incorporates four distinctive search mechanisms. They comprise a nonlinear exploration scheme for dynamic search territory adjustment, a chaotic leadership dispatching strategy among the dominant wolves, a rectified spiral local exploitation action, as well as probability distribution-based leader enhancement. The evolving CNN-LSTM models are subsequently devised using the proposed GWO variant, where the network topology and learning hyperparameters are optimized for time series prediction and classification tasks. Evaluated using a number of benchmark problems, the proposed GWO-optimized CNN-LSTM models produce statistically significant results over those from several classical search methods and advanced GWO and Particle Swarm Optimization variants. Comparing with the baseline methods, the CNN-LSTM networks devised by the proposed GWO variant offer better representational capacities to not only capture the vital feature interactions, but also encapsulate the sophisticated dependencies in complex temporal contexts for undertaking time-series tasks

Object recognition using enhanced particle swarm optimization.

The identification of the most discriminative features in an explainable AI decision-making process is a challenging problem. This research tackles such challenges by proposing Particle Swarm Optimization (PSO) variants embedded with novel mutation and sampling iteration operations for feature selection in object recognition. Specifically, five PSO variants integrating different mutation and sampling strategies have been proposed to select the most discriminative feature subsets for the classification of different objects. A mutation strategy is firstly proposed by randomly flipping the particle positions in some dimensions to generate new feature interactions. Moreover, instead of embarking the position updating evolution in PSO, the proposed PSO variants generate offspring solutions through a sampling mechanism during the initial search process. Two offspring generation sampling schemes are investigated, i.e. the employment of the personal and global best solutions obtained using the mutation mechanism, respectively, as the starting positions for the subsequent search process. Subsequently, several machine learning algorithms are used in conjunction with the proposed PSO variants to perform object classification. As evidenced by the empirical results, the proposed PSO variants outperform the original PSO algorithm, significantly, for feature optimization

Feature selection using enhanced particle swarm optimisation for classification models.

In this research, we propose two Particle Swarm Optimisation (PSO) variants to undertake feature selection tasks. The aim is to overcome two major shortcomings of the original PSO model, i.e., premature convergence and weak exploitation around the near optimal solutions. The first proposed PSO variant incorporates four key operations, including a modified PSO operation with rectified personal and global best signals, spiral search based local exploitation, Gaussian distribution-based swarm leader enhancement, and mirroring and mutation operations for worst solution improvement. The second proposed PSO model enhances the first one through four new strategies, i.e., an adaptive exemplar breeding mechanism incorporating multiple optimal signals, nonlinear function oriented search coefficients, exponential and scattering schemes for swarm leader, and worst solution enhancement, respectively. In comparison with a set of 15 classical and advanced search methods, the proposed models illustrate statistical superiority for discriminative feature selection for a total of 13 data sets

Improving K-means clustering with enhanced Firefly Algorithms

In this research, we propose two variants of the Firefly Algorithm (FA), namely inward intensified exploration FA (IIEFA) and compound intensified exploration FA (CIEFA), for undertaking the obstinate problems of initialization sensitivity and local optima traps of the K-means clustering model. To enhance the capability of both exploitation and exploration, matrix-based search parameters and dispersing mechanisms are incorporated into the two proposed FA models. We first replace the attractiveness coefficient with a randomized control matrix in the IIEFA model to release the FA from the constraints of biological law, as the exploitation capability in the neighbourhood is elevated from a one-dimensional to multi-dimensional search mechanism with enhanced diversity in search scopes, scales, and directions. Besides that, we employ a dispersing mechanism in the second CIEFA model to dispatch fireflies with high similarities to new positions out of the close neighbourhood to perform global exploration. This dispersing mechanism ensures sufficient variance between fireflies in comparison to increase search efficiency. The ALL-IDB2 database, a skin lesion data set, and a total of 15 UCI data sets are employed to evaluate efficiency of the proposed FA models on clustering tasks. The minimum Redundancy Maximum Relevance (mRMR)-based feature selection method is also adopted to reduce feature dimensionality. The empirical results indicate that the proposed FA models demonstrate statistically significant superiority in both distance and performance measures for clustering tasks in comparison with conventional K-means clustering, five classical search methods, and five advanced FA variants

The value of carcinoembryonic antigen stage in staging, prognosis, and management of colorectal cancer: results from two cohort studies

BackgroundCombining the carcinoembryonic antigen (CEA) level (C stage) with TNM staging can provide a more comprehensive prognostic assessment of colorectal cancer (CRC). However, the clinical value of incorporating CEA status into the TNM staging system needs to be evaluated.MethodsWe used the SEER database (N = 49,350) and a retrospective cohort from China (N = 1,440). A normal CEA level was staged as C0 and an elevated CEA level was staged as C1. Restricted cubic spline analysis was used to examine the dose-response relationship between the CEA level and survival. The Kaplan-Meier method with the log-rank test was used to plot survival curves. Multivariable Cox proportional hazards regression models with forward stepwise variable selection were used to estimate the hazard ratios and 95% confidence intervals.ResultsPatients with C1 were more likely to have advanced disease than those with C0. CEA on a continuous scale was positively associated with mortality risk. Compared with patients with C0 stage, those with C1 stage had significantly lower survival rates. In the SEER dataset, C1 was independently associated with poor prognosis in patients with CRC, with an approximately 70% increased risk of mortality. Patients with C1 stage had significantly lower survival than those with C0 stage at all clinical stages. Incorporating the C stage into the TNM staging refined the prediction of prognosis of patients with CRC, with a gradual decline in prognosis from stage I C0 to stage IV C1. A similar pattern was observed in the present retrospective cohort study. At each lymph node stage, patients with C1 had significantly lower 5-year survival rates than patients with C0. Compared with lymph node positivity, CEA positivity may have a stronger correlation with a worse prognosis.ConclusionOur findings not only validated the independent prognostic significance of CEA in CRC but also demonstrated its enhanced prognostic value when combined with TNM staging. Our study provides evidence supporting the inclusion of C stage in the TNM staging system

Serum creatinine/cystatin C ratio as a prognostic indicator for patients with colorectal cancer

BackgroundThis study aimed to explore the relationship between creatinine/cystatin C ratio and progression-free survival (PFS) and overall survival (OS) in colorectal cancer (CRC) patients undergoing surgical treatment.MethodsA retrospective analysis was conducted on 975 CRC patients who underwent surgical resection from January 2012 to 2015. Restricted three-sample curve to display the non-linear relationship between PFS/OS and creatinine-cystatin C ratio. Cox regression model and Kaplan-Meier method were used to evaluate the effect of the creatinine-cystatin C ratio on the survival of CRC patients. Prognostic variables with p-value ≤0.05 in multivariate analysis were used to construct prognostic nomograms. The receiver operator characteristic curve was used to compare the efficacy of prognostic nomograms and the traditional pathological stage.ResultsThere was a negative linear relationship between creatinine/cystatin C ratio and adverse PFS in CRC patients. Patients with low creatinine/cystatin C ratio had significantly lower PFS/OS than those with high creatinine/cystatin C ratio (PFS, 50.8% vs. 63.9%, p = 0.002; OS, 52.5% vs. 68.9%, p < 0.001). Multivariate analysis showed that low creatinine/cystatin C ratio was an independent risk factor for PFS (HR=1.286, 95%CI = 1.007–1.642, p=0.044) and OS (HR=1.410, 95%CI=1.087–1.829, p=0.010) of CRC patients. The creatinine/cystatin C ratio-based prognostic nomograms have good predictive performance, with a concordance index above 0.7, which can predict the 1–5-year prognosis.ConclusionCreatinine/cystatin C ratio may be an effective prognostic marker for predicting PFS and OS in CRC patients, aid in pathological staging, and along with tumour markers help in-depth prognostic stratification in CRC patients

A Pilot Study on the Impacts of Lung-Strengthening Qigong on Wellbeing

Background Qigong embraces a range of self-care exercises originating from China. Lung-Strengthening Qigong (LSQ) is a specific technique for maintaining and improving physical and mental wellbeing. Methods We recruited 170 practitioners and 42 non-practitioner/control samples to investigate the impacts of LSQ practice on body, mind, thoughts, and feelings. This is a pilot study pursued to plan for an adequately powered, non-clinical randomized controlled trials (RCT) on overall wellbeing and health and to evaluate the adequacy of delivering the physical activity intervention with fidelity. Self-evaluation-based data collection schemes were developed by regularly requesting completion of a questionnaire from both practitioner and control group, and an online diary and end of study survey (EOS) completion only from the practitioners. Diverse types of analyses were conducted, including statistical tests, machine learning, and qualitative thematic models. Results We evaluated all different data resources together and observed that (a)the impacts are diverse, including improvements in physical (e.g., elevated sleep quality, physical energy, reduced fatigue), mental (e.g., increased positivity, reduced stress), and relational (e.g., enhanced connections to self and nature) wellbeing, which were not observed in control group; (b)measured by the level-of-effectiveness, four distinct clusters were identified, from no-effect to a high-level of effect; (c)a majority (84 %) of the LSQ practitioners experienced an improvement in wellbeing; (d)qualitative and quantitative analyses of the diary entries, questionnaires, and EOS were all found to be consistent, (e)majority of the positively impacted practitioners had no or some little prior experience with LSQ. Conclusions Novel features of this study include (i)an increased sample size vis-à-vis other related studies; (ii)provision of weekly live-streamed LSQ sessions; (iii)integration of quantitative and qualitative type of analyses. The pilot study indicated that the proportion of practitioners who continued to engage in completing the regular-interval questionnaires over time was higher for practitioners compared to the control group. The engagement of practitioners may have been sustained by participation in the regular live LSQ sessions. To fully understand the impacts of LSQ on clinical/physiological outcomes, especially for specific patient groups, more objective biomarkers (e.g. respiratory rate, heart rate variation) could be tracked in future studies

Comprehensive comparative analysis of prognostic value of serum systemic inflammation biomarkers for colorectal cancer: Results from a large multicenter collaboration

BackgroundThe incidence of colorectal cancer (CRC) is common and reliable biomarkers are lacking. We aimed to systematically and comprehensively compare the ability of various combinations of serum inflammatory signatures to predict the prognosis of CRC. Moreover, particular attention has been paid to the clinical feasibility of the newly developed inflammatory burden index (IBI) as a prognostic biomarker for CRC.MethodsThe discrimination capacity of the biomarkers was compared using receiver operating characteristic curves and Harrell’s C-index. Kaplan-Meier curves and log-rank tests were used to compare survival differences between the groups. Cox proportional hazard regression analysis was used to determine the independent prognostic factors. Logistic regression analysis was used to assess the relationship between IBI, short-term outcomes, and malnutrition.ResultsIBI had the optimal prediction accuracy among the systemic inflammation biomarkers for predicting the prognosis of CRC. Taking IBI as a reference, none of the remaining systemic inflammation biomarkers showed a gain. Patients with high IBI had significantly worse overall survival than those with low IBI (56.7% vs. 80.2%; log-rank P<0.001). Multivariate Cox regression analysis showed that continuous IBI was an independent risk factor for the prognosis of CRC patients (hazard ratio = 1.165, 95% confidence interval [CI] = 1.043–1.302, P<0.001). High IBI was an independent risk factor for short-term outcomes (odds ratio [OR] = 1.537, 95% CI = 1.258–1.878, P<0.001), malnutrition (OR = 2.996, 95% CI = 1.471–6.103, P=0.003), and recurrence (OR = 1.744, 95% CI = 1.176–2.587, p = 0.006) in CRC patients.ConclusionsIBI, as a reflection of systemic inflammation, is a feasible and promising biomarker for assessing the prognosis of CRC patients